Large-scale Crowdsourced Study for High Dynamic Range Pictures

Measuring digital picture quality, as perceived by human observers, is increasingly important in many applications in which humans are the ultimate consumers of visual information. Standard dynamic range (SDR) images provide 8 bits/color/pixel. High dynamic range (HDR) images, usually created from multiple exposures of the same scene, can provide 16 or 32 bits/color/pixel, but need to be tonemapped to SDR for display on standard monitors. Multi-exposure fusion (MEF) techniques bypass HDR creation and fuse exposure stack directly to SDR images with aesthetically pleasing luminance and color distributions. Many HDR and MEF databases have a relatively small number of images and human opinion scores. The opinion scores have been obtained in stringently controlled environments, thereby limiting realistic viewing. To overcome these challenges, we have conducted a massively crowdsourced online subjective study. The primary contributions of this paper are (1) creating the ESPL-LIVE HDR Image Database containing diverse images obtained by TMO and MEF algorithms, with and without post-processing; (2) conducting a large-scale subjective study using a crowdsourced platform to gather more than 300,000 opinion scores on 1,811 images from over 5,000 unique observers; and (3) evaluating correlation performance of stateof-the-art no-reference image quality assessment algorithms vs. opinion scores on these images. The database is available at: http://signal.ece.utexas.edu/%7Edebarati/HDRDatabase.zip.